.NET: CI hardening — split Functions tests, re-enable skipped integration tests#5717
Conversation
- Add -TestProjectNameExclude parameter to New-FilteredSolution.ps1 - Add 'functions' and 'core' path filters to paths-filter job - Exclude DurableTask/AzureFunctions from main dotnet-test job - Remove emulator setup from dotnet-test (no longer needed) - Add new dotnet-test-functions job (ubuntu/net10.0 only, path-conditional) - Update merge gate and report job to include dotnet-test-functions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
changed the title
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 91%
✓ Correctness
The PR correctly splits DurableTask/AzureFunctions integration tests into a dedicated CI job. The path filters, conditional logic, merge gate updates, and PowerShell script changes are all properly implemented. The new TestProjectNameExclude parameter correctly iterates over patterns and is applied after TestProjectNameFilter. The merge gate handles skipped jobs correctly since 'skipped' results don't trigger the failure/cancelled checks. The report job's artifact download pattern correctly matches the new artifact name.
✓ Security Reliability
This PR is a clean CI refactoring that splits DurableTask/AzureFunctions integration tests into a dedicated job. The merge gate correctly handles skipped jobs (result='skipped' doesn't match 'failure' or 'cancelled' checks), the PowerShell exclusion parameter is properly implemented, secrets are used consistently with existing patterns, and path filters appropriately trigger the new job when core dependencies change. No security or reliability issues identified.
✓ Test Coverage
This PR is a CI infrastructure change that splits DurableTask/AzureFunctions integration tests into a dedicated job and adds a -TestProjectNameExclude parameter to New-FilteredSolution.ps1. The new parameter has implicit functional coverage through the CI workflow (the filtered solution is used by dotnet test, so failures would surface). However, there are no dedicated unit tests (Pester tests) for the script — this is a pre-existing gap, not introduced by this PR. The existing parameters (e.g., -TestProjectNameFilter) also lack isolated tests. The workflow logic is sound: path filters, merge gate inclusion, and artifact naming all align correctly.
✗ Design Approach
The split-job approach is mostly sound, but the new path gating for
dotnet-test-functionsmisses part of the DurableTask/AzureFunctions dependency chain. As written, a change limited toMicrosoft.Agents.AI.Workflows.Generatorscan alter those integration-test binaries while causing the dedicated functions job to skip, and the maindotnet-testjob now excludes those same integration tests.
Flagged Issues
- The new
corepath filter in.github/workflows/dotnet-build-and-test.ymlomitsdotnet/src/Microsoft.Agents.AI.Workflows.Generators/**, even though DurableTask depends onMicrosoft.Agents.AI.Workflows(dotnet/src/Microsoft.Agents.AI.DurableTask/Microsoft.Agents.AI.DurableTask.csproj:30) andMicrosoft.Agents.AI.WorkflowsincludesMicrosoft.Agents.AI.Workflows.Generators.csprojas analyzer (dotnet/src/Microsoft.Agents.AI.Workflows/Microsoft.Agents.AI.Workflows.csproj:34-39). With the new exclusion of DurableTask/AzureFunctions ITs from the main job, generator-only changes can now skip all affected integration tests.
Suggestions
- Expand the
corepaths-filter block to includedotnet/src/Microsoft.Agents.AI.Workflows.Generators/**source-generator changes still triggerdotnet-test-functionsfor the DurableTask/AzureFunctions dependency chain.
Automated review by giles17's agents
There was a problem hiding this comment.
Pull request overview
Splits the DurableTask and Azure Functions integration tests out of the main .github/workflows/dotnet-build-and-test.yml dotnet-test job into a dedicated CI job to avoid emulator setup overhead in the primary test matrix and to enable more granular path-based job triggering.
Changes:
- Added
-TestProjectNameExcludesupport todotnet/eng/scripts/New-FilteredSolution.ps1to remove specific test projects after applying-TestProjectNameFilter. - Extended
paths-filterwithfunctionsandcorefilters and used them to conditionally run a newdotnet-test-functionsjob (ubuntu/net10.0 only). - Excluded DurableTask/AzureFunctions integration test projects from the main integration-test solution generation and removed the emulator setup from the main
dotnet-testjob.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| dotnet/eng/scripts/New-FilteredSolution.ps1 | Adds an exclusion parameter to filter out specific test projects when generating filtered .slnx files. |
| .github/workflows/dotnet-build-and-test.yml | Introduces path filters and a new dedicated job to run DurableTask/AzureFunctions integration tests with emulator setup. |
…tnetChanges gate from functions job Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes MissingMethodException on WebSearchToolResultContent.get_Results() caused by Anthropic 12.13.0 being compiled against an older Microsoft.Extensions.AI.Abstractions version. Suppress RT0003 in AI.Abstractions.csproj as the transitive reference from the upgraded Anthropic SDK conflicts with the explicit one. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TestProjectNameFilter -> TestProjectNameIncludeFilter TestProjectNameExclude -> TestProjectNameExcludeFilter Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The RT0003 suppression was added during the Anthropic SDK 12.20.0 upgrade but the warning no longer fires. Removing it to keep the NoWarn list minimal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ation-tests # Conflicts: # dotnet/tests/AnthropicChatCompletion.IntegrationTests/AnthropicSkillsIntegrationTests.cs
Both our branch and main added WebhookKey to the Anthropic test mock classes, resulting in CS0102 duplicate definition errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Motivation and Context
Improve CI signal quality by splitting long-running integration tests into a dedicated job and re-enabling integration tests that were unnecessarily skipped.
Description
CI job split:
-TestProjectNameExcludeparameter toNew-FilteredSolution.ps1functionsandcorepath filters to thepaths-filterjobdotnet-testjobdotnet-test(no longer needed)dotnet-test-functionsjob (ubuntu/net10.0 only, path-conditional)dotnet-test-functionsRe-enable Anthropic integration tests:
Re-enable CheckSystem declarative integration tests:
DeclarativeWorkflowTest.csandDeclarativeCodeGenTest.csCheckSystem.jsontest case: setmin_response_count: 1,max_response_count: 1, and explicitmin/max_message_count: 0to match SendActivity behavior (response events emitted but no messages stored)Re-enable Foundry OpenAPI integration test:
Skip = "For manual testing only"fromAsAIAgent_WithOpenAPITool_NativeSDKCreation_InvokesServerSideToolAsyncRetryFact(3 retries, 5s delay)for transient failure resilienceContribution Checklist